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Abstract 

The co-word problem of a group G generated by a set X is defined as 
the set of words in X which do not represent 1 in G. We introduce a new 
method to decide if a permutation group has context-free co-word prob- 
lem. We use this method to show, that the Higman-Thompson groups, 
and therefore the Houghton groups, have context-free co-word problem. 
We also give some examples of groups, that even have an easier co-word 
problem. We call this property semi-deterministic context-free. The sec- 
ond Houghton group belongs to this class. 



1 Introduction 

Let G be a group with a finite generating set X . The word problem of G with 
respect to X, denoted by W(G, X), is the set of all words in (X ± )* which repre- 
sent the identity in G. By coW(G, X) we denote the complement of W(G,X), 
i.e. the set of words in X ± which do not represent 1 in G. coW(G, X) is called 
the co-word problem of G with respect to X . We say the group G has a context- 
free word problem (resp. co-word problem), or equivalently belongs to the class 
CF (resp. coCF), if and only if W(G,X) (resp. coW(G, X)) is a context-free 
language. This makes sense, since these questions are independent of the choice 
of generators, as was shown by Muller and Schupp in ^U] resp. Holt, Rees, 
Rover and Thomas in |7j. As a consequence we can always choose a generating 
set, which can easily be handled. 

It is a well-known fact, that context-free languages are precisely the lan- 
guages which can be recognized by a non-deterministic pushdown automaton 
(NPDA). For a definition and details see e.g. The groups of class CF are 
classified to be the virtual free groups by Muller and Schupp see (^DJ, [3]). 
Thereby groups of class CF can already be recognized by deterministic push- 
down automata. 

The groups of class coCF were first studied by Holt, Rover, Rees and Thomas 
in [2| . If a deterministic automaton recognizes a language, then there exists an 
automaton, which recognizes the complement, so the class CF is contained in the 
class coCF. Further it is shown, that the class coCF is closed under taking direct 
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products, restricted standard wreath products with CF-top group, and passing 
to finitely generated subgroups and finite index overgroups. They also remarked, 
that there are no further examples of coCF groups known and conjecture that 
the class coCF is not closed under taking free products. 

The Houghton groups H n , which are defined in Section [3 do not arise out 
of these constructions, so they are indeed new examples. In Rover showed, 
that all Houghton groups are subgroups of the Higman-Thompson groups G n ^ r , 
which due to Holt and Rover jS] have indexed co-word problem, a similar but 
weaker property. There is no group known, which has indexed co-word problem 
but does not belong to coCF. The Houghton groups were untruly suspected to 
be such groups. In contrary we will show in Section |SJ 

Theorem 1. The Higman-Thompson groups G n ,r are coCF. 

Corollary 2. Letn>2. Then H n is coCF. 

Also in Section |21 we will define the groups Hou(G), which are permutation 
groups similar to the Houghton groups. In fact Hou(Z) is isomorphic to H 2 . 

We will show, that Hou(F n ) is coCF. This result would also follow from the 
construction in section 0] but we show a stronger result in section [3J There is a 
class of languages in beetween nondeterministic and deterministic context-free 
languages. This class is recognized by PDA's that may write non-deterministi- 
cally a word onto the stack before the beginning of the computation and then 
behave totally deterministic. In the case of languages it is easy to see that this 
class is a true sub-class of the context-free languages, and it seems to be very 
reasonable that coW(iJ3) is not of that type, but we don't know how to prove 
it. We will call this class semi-deterministic context-free languages, and obtain: 

Theorem 3. coW(Hou(J 7 ' n )) is semi-deterministic context-free. 

This property is also independent of the choice of generators. This implies 
that finitely generated subgroups of semi-deterministic context-free subgroups 
inherit this property. A careful analysis of the proofs in [J] shows, that they 
easily translate into proofs for semi-deterministic co-word problems. Thus this 
property is also closed under taking direct products, finite index overgroups and 
restricted standard wreath products with CF-top groups. 

In section [S] we discuss further groups whose membership to coCF can be 
established by our method. Throughout this work we will make frequent use 
of the fact, that group elements are represented by words in the generators. 
Thereby we will (abusing notation) not distinguish beetween words and group 
elements. 

2 Definition of Houghton groups and Higman- 
Thompson groups 

Definition 4. Let Y = (V, E) be a locally finite graph. A quasi- automorphism 
of r is a permutation of V , which respects all but finitely many adjacencies. 
The group of all quasi-automorphisms is denoted by QAut(r). 
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Figure 1: Two possible sets of generators of the Houghton group H3. All ele- 
ments act as shifts, where the arrows define, which rays shifts into each other. 
While the set {x,y,z} is natural for the set {si, 52,53} is natural for * 3 . 
They transform into each other as follows: r = x, s = [x, y]y, t = z. 

Note that QAut(r) does not change, if we change Y by adding or removing 
finitely many edges. Thus if T is a finite graph, QAut(r) is just the symmetric 
group on V which we denote by Sy- Let No be the graph with non-negativ 
integer vertices and {i, j} is an edge if and only if i = j ± 1. The disjoint union 
of n copies of No is called the n-star *g. We denote the points of *g by a pair 
(k, I), which is the vertex k of the Ith copy of No (see figure QJ. For each quasi- 
automorphism $ of * l there exist r > 0,si...,s n e Z and p> £ S n , such that 
for all Z > r 

*o(k,l) = {k + ai,<p(l)). 

So in the complement of a sufficiently large set the copies of No are permuted 
and the points are just shifted. Clearly the sum of the Sj has to be 0. 

The nth Houghton group H n is classically defined as the subgroup of all 
elements $ of QAut(*o) f° r which this element ip is the identity of S n . Since 
= n\ we obtain that H n is a subgroup of finite index in QAut(*g)- In 
particular QAut(*g) is coCF if and only if H n is coCF. 

As observed obove the group QAut(*g ) does not change if we add edges from 
the of the first copy of No to all other 0's, and reenumerate * l by adding 1 to 
each vertex, which does not belong to the first copy. The result is called *" . For 
technical reasons, we will use *" instead of using the usual definition. Thereby 
we denote the points of *" by and (fc, I), k € N, I G {1, . . . , n} with the above 
notation. Be aware, that the shifts in this model, which we define afterwards, 
are not exactly the same, as in the standard model (see figure QJ. 

Let n > 1. All H n are finitely generated (In fact Brown showed in [2J that 
H n is of type FP„_i, but not of type FP„). The shift Sj £ H n acts on *„ as 
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follows (rays modulo n): 

{(k,l) + 1 

(k-l,l) l = i,k>2 
l = i, k = l 

(k + l,l) l = i + l 

and Si(0) = 1)- For n > 3 the commutator [si, s;+i] acts as the trans- 

position of and So the group generated by the shifts contains all finite 

permutations and is thereby equal to H n . 

A special case is H 2 , which needs an additional generator. Because * 2 is just 
the Cay ley graph of Z generated by X = {1}, we will treat it as a special case 
of the following definition. 

Definition 5. Let G be a group finitely generated by X. With Hou(G) we 
denote the group of those quasi- automorphims of the Cayley graph of (G, X) 
which act on all but finitely many vertices like a left-multiplication with an 
element of G. In other words Hou(G) — Sq * G. Here Sq denotes the group of 
all permutations with finite support. 

Note that this definition is independent of the choice of generators. 

If G is generated by X = {xi, . . . , x n } then Hou(G) is generated by X = 
{x\, . . . x n , <ti, . . . , <r n }. Here <Ji is the quasi-automorphism which maps 1 to Xi, 
Xi to 1 and stabilizes all other points. Thus H 2 = Hou(Z) is generated by 2 
elements. 

From a related concept the Higman- Thompson groups arise. Fix two integers 
n > 2, r > 1 and let Q = {qi . . . q r } and £ = {a± . . . a n } be two finite sets. Let 
£1 = QE N be the set of infinite sequences starting with some followed by 
elements of S. A barrier is a subset B C QT,* such that for every element 
uj € fl there is exactly one b € B with u> € 6S N , i.e. b is the unique prefix of u 
in B. 

Two barriers Bi,B 2 of the same cardinality and a bijection (f> : B\ — > B 2 
induce a bijection g^ : fl — > £1 by prefix replacement. The set of all such 
induced bijections forms a group, the Higman-Thompson group G n ^ r . For a 
more detailed description of G„, r see [TT] . 

3 Hou(F n ) is semi-deterministic coCF 

In order to prove Theorem 2, it will suffice to construct a PDA, which recognizes 
coW(Hou(F„)). To demonstrate the way this automaton works, we first solve 
the co- word problem of H 2 ■ 

Theorem 6. Let X be the generating set of H 2 consisting of X = {t, t}, with 
t = «-> 1 and t = (i i— ► i + 1). The Houghton group H 2 is semi- deterministic 
coCF with respect to the generating set X . 
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Proof. We will use the following idea. An element g G H2 is a pair (a, s) with 
a G Soo and s G Z (where we make use of the fact, that if 2 = X Z), hence 
g is nontrivial if and only if a is a nontrivial permutation or s / 0. 

The NPDA P, which recognizes CoW(G, X), first guesses non-determinis- 
tically whether to check the non-triviality of s or the nontriviality of a. In order 
to check the non-triviality of s, all P has to do, is to check that the exponent- 
sum of t is distinct from 0. This clearly can be done by a deterministic PDA 
/', 

Now we describe, how an automaton P$ decides, if a an element of the form 
(er, 0) is nontrivial. An element a is nontrivial if and only if there is a k G Z 
with cr(fc) ^ k. So all Ps has to do, is to guess non-deterministically for which 
k this condition is tested. 

Ps consists of four states called / (the initial state), +, — , and qA (the accept 
state). Let V = {A : B,C,D,jf} be the stack alphabet, where # is the end-of- 
stack symbol. In the beginning, Ps writes a random number of A's (for k > 0) 
or B's (for k < 0) onto the stack. The further behavior of the automaton is 
completely deterministic. Instead of describing the transition matrix, we explain 
the main idea. The stack 'remembers' the image of the element k, which equals 
the number of A's (resp. J5's) that were written onto the stack in the beginning. 
So roughly speaking t will add an A and t^ 1 will add a B. A t only changes 
the stack, when the stack contains at most one A and no B. 

Below these A's and B's the automaton remembers the number of r with 
an effect on the given start-element by C's, when the element is moved to the 
right and by D's, when its moved to the left. This is possible, because r only 
has an effect on our element, when the stack contains at most one A. When the 
word is read, all the A's and B's are dropped out of the stack and the word is 
accepted if C's or D's remain. Clearly, during this process A and B cancel each 
other, so do C and D. 

The explicit proof can be found in figure^ which shows the transition matrix 
of this automaton. □ 

Proof of Theorem^ For Hou(F„) = Sp n * F n we choose the set of generators 
{xi, . . . , x n , <7i, . . . , cr n }. The letters Xi correspond to the generators of the free 
group and act by (left-) multiplication on the Cayley-graph of F n and is the 
switch 1 <-> Xi. 

The PDA which recognizes coW(Hou(i 7 '„)) now does nearly the same as the 
one that recognizes coW(i?2)- First it guesses whether it checks the F n part or 
the permutation part. The deterministic PDA, which checks the F n part only 
has to write the x^-part of the word onto the stack and freely reduce whenever 
possible. 

To check the permutation part the automaton guesses a non-fixpoint and 
writes it onto the stack. Now the word operates letter by letter. The o~i only 
have an effect if the stack contains at most one letter. So below the position of 
the point, the effect of the Oi can be stored on the stack by a reduced word in 
n . In the end the automaton drops the position and if a r remains, the word is 
non-trivial. Further details are left to the reader. □ 
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Figure 2: The transition matrix of the automaton, which recognizes coW(i?2)- 
$ denotes a symbol, that indicates the end of the input string. Clearly the states 
+ and — are only needed to get account to the second level of the stack. An 
automaton with a look-ahead of 1 would not need them. 

4 Houghton groups are coCF 

In order to show that the Houghton groups and later on the Higman-Thompson 
groups are coCF we need a technical lemma. As C.E. Roever pointed out to us, 
the result is already proved in so we suppress the proof 

Lemma 7. Let L be a context-free language, then L° := {yx\x,y S € 
L}, the set of all cyclic permutations of the words in L, is also a context-free 
language. 

Using this we are able to proof that the Houghton groups are coCF. Despite 
the fact that this statement follows from Theorem ^ we carry out the proof 
explicitly, as it serves as an illustration of the proof of Theorem ^ 

Proof of Corollary\^ The Houghton group H n for n > 3 is generated by the 
shifts si, . . . , s n as defined above. So Sj shifts ray i into ray (i + 1). 

Let £ = {sf , . . . , s^}, and L = {w <E £*|k;(0) ^ 0} be the set of words for 
which is not a fixpoint. 

We only need to verify two things now: L is context free and L° = coW(H n ), 
that is every nontrivial word in the generators is a cyclic permutation of a word 
in L. 

Step 1: L° = coW(H n ). 
'C' is obvious since conjugation does not trivialize an element. 
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'D' Let w be a word in coW(H n ). We need to show there are words x, y with 
w = xy and a non fixpoint p of w, such that x(p) = 0. Let q = (91,52) be a 
non-fixpoint. If there is a prefix x of u> with a;(g) = 0, we are done. If not, w 
acts far outside on the ray (•, (72) like a shift. So there exists also a ray (•, k), on 
which io acts like an outbound shift (again only far outside). But then there is 
at least one point p = (pi,P2),P2 7^ k whose image is on ray (•, k). This implies 
the existence of a prefix x of w with x(p) = 0. 

Step 2: L is context-free. 
L is even a one counter language. In one state one can memorize on which ray 
has currently been permuted, and on the counter one can memorize how far. 

This completes the proof. □ 

The group Hqo is not finitely generated, so it cannot be coCF. Let H.^ = 
Hoo x Z, where a G Z acts on *°° as the rotation which maps ray i to ray i + a. 
TL is coCF and thereby is locally coCF. The proof is in the same manner 
and left to the reader. 



5 The Higman-Thompson groups G n iT 

Using the method of Section [5] we are now able to proof Theorem ^ 

Proof of Theorem^ Higman showed in [5] that G n , r is finitely presented. Let 
X be a finite set of generators for G n ^ r . For each element r G G n ^ r there exists 
an k T such that for all u> S f2 there exist it, u' S QS*, iff(u) < k T and a/ G S* 
with uu' = a; and t(lu) — u'uj' . Here lg(u) denotes the length of u. This means, 
that sequences are only changed on prefixes shorter than k T . We observe that r 
induces a map from QS feT E* (all finite strings with length greater than k T ) to 
QY,* . Let k = max xe xk x . 

Let M = {qwo\0\ . . . \q G G be the set of all infinite sequences 

for which all entries greater than k are equal to a\. Obviously \M\ = rn k and 
hence M is finite. 

Let L be the set of words in X*, which do not fix all sequences in M . Again 
two things remain to be shown. L is context free and L° = coW(G ntr ): every 
non-trivial word in the generators is a cyclic permutation of a word in L. 

Step 1: L° = coW(G n>r ). 
For every word z in coW(L) we have to show the existence of a non-fix-sequence 
which during the action letter- by-letter is mapped onto a sequence of M. Ob- 
viously z has a non-fix-sequence. Let r z = qa^ , Ui 2 ... be such a sequence. Let 
m > k be the minimal number such that no prefix of z induces a map which 
sends the string qai ± , . . . o~i m to a string of length less than k. Because of the 
minimality of m there exists a prefix x of z which maps qo~i x , . . . o~i m to a string 
s with length exactly k. By the construction of k all sequences with prefix 
qai 1 ,...Ui m are mapped onto a sequence with the same prefix as z(r z ) and 
hence are non- fix-sequences, especially the sequence to = qa^ . . . o im o\a\ 
But x(lo) = Sa\a\ . . . G M . This proves step 1. 

Step 2: L is context-free. 



7 



Let oj — qwo\<j\ ... be an arbitrary element of M. Since M is finite, it is 
sufficient to show, that the language V of all words in X* , which do not fix 
ui is context-free. We describe an automaton with an access depth of k (the 
automaton is allowed to read and write in the k top levels of the stack), which 
recognizes L': 

The computation starts with the word qwa\#, where / is a random non- 
negative integer, on the stack (q on the top). Every letter changes the fc-prefix 
in the manner the group-action defines. If the end-of-stack-symbol # is ever 
visible to the automaton, it rejects. The word is accepted, if the computation 
ends with gui<7™# on the stack for some m. This can be recognized by the 
automaton. □ 

6 Towards the limits 

Until now we have not reached the limits of our methods. Because of the 
statements in the introduction we have also seen, that QAut(* n ) is coCF. The 
arguments of section 0] are clearly strong enough, to show: 

Theorem 8. The following holds: 

1. Hou(G) = Sg x G is coCF if and only if G is coCF. 

2. QAut(Z n ) is coCF. 

We further conjecture, that QAut(G) is coCF if and only if G is coCF and 
QAut(G) is finitely generated. The motivation for a further study of QAut(r) 
groups comes from the fact, that every finitely generated group is a subgroup 
of QAut(r)-groups. The challenge is just to find a handsome graph T. 

Another reason for a further study of the Higman-Thompson groups is the 
interesting result of J.C. Birget [Q, which shows the existence of a subgroup 
A < G3.1 and a semidirect product B = A x Z such that the co-word problem 
of B is AP-complete and Theorem ^implies, that coW(G„,r) and the co-word 
problem of all subgroups can be solved in cubic time. 
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